As enterprises begin to integrate more of their data systems, complex server applications are used to provide access to the data systems. The server applications allow users to access, modify, and operate an enterprise's data systems. In order to be effective, the server applications must allow a large number of users to access the data systems at the same time. Large server applications must be able to handle thousands of users simultaneously and with minimal processing delay. To fulfill these requirements, large server applications demand enormous computing resources.
The processing requirements of large server applications may be satisfied by employing a distributed computing environment. A distributed computing environment is comprised of a group of networked computers working in concert to provide a variety of computing services. In this type of computing environment, software applications are separated into many application components. These components are distributed across the network of computers, so that different components may be executed by different computers within the distributed environment. The application components are designed to effectively operate in a distributed environment. The advantage of designing applications for a distributed computing platform include increased scalability to accommodate a large numbers of users, increased reliability, and efficient use of computing resources.
One example of a distributed computing environment is the .NET™ architecture developed by Microsoft™. The .NET™ architecture uses XML web services as small, discrete building-block applications connected to each other and other larger applications through a network or the Internet. XML web services may be distributed among one or more client or server computers. XML web services employ the extensible markup language (XML) for communication and may be written in a variety of programming languages, including the C# programming language developed by Microsoft
Another example of a distributed computing environment is the Java™ 2 platform, Enterprise Edition (J2EE) architecture, which was developed by Sun Microsystems, Inc. J2EE applications are comprised primarily of Enterprise Java™ Beans (EJB), which are self-contained, reusable application components written in the Java™ programming language. The specification for the J2EE architecture is described in Java™ 2 Platform Enterprise Edition Specification, v1.3, available at Java.sun.com/j2ee/j2ee-1—3-pfd4-spec.pdf and incorporated by reference herein.
In the J2EE architecture, server applications are executed within an application server program installed on each computer in a distributed computing environment. Each server application may be spread over many application servers within the distributed computing environment, or concentrated within a single application server located on a single computer. Each computer may contain multiple processors to further distribute the workload of the application server program.
The J2EE application server program provides low-level functionality needed by server applications in a distributed programming environment. The functions provided by the J2EE application server program enable the application server to manage system resources, threads, memory, database connections, security, client connectivity, transaction management, and communications between users and applications.
The use of the Java™ programming language is one advantage of the J2EE architecture. The Java™ language provides a platform-independent execution environment, which enables complex applications to run effectively and efficiently on a variety of different types of computing devices. At the core of this platform independence is the Java™ virtual machine. The Java™ virtual machine provides the interface between the abstract Java™ environment and the native system. Although different native systems will require different Java™ virtual machines, each Java™ virtual machine will be able to run the same Java™ application.
The Java™ virtual machine is an abstract or “virtual” computer. Generally, the Java™ virtual machine is implemented as a software program executing on top of a native or “real” hardware and operating system, referred to as a native system. The Java™ virtual machine executes Java™ applications by translating the compiled Java™ code, referred to as bytecodes, into a format suitable for execution on the native system. The specification of the Java™ virtual machine is described in the Java™ Virtual Machine Specification, Second Edition, which is available at Java.sun.com/docs/books/vmspec/2nd-edition/html/VMSpecTOC.doc.html and is incorporated by reference herein.
A Java™ virtual machine may be implemented in a variety of ways. A simple Java™ virtual machine uses a Java™ interpreter. A Java™ interpreter translates a Java™ program as it is executed. A Java™ interpreter executes a Java™ program by retrieving a bytecode from an application, selecting a pre-existing native code sequence that performs the action of that bytecode for execution, and dispatching execution to the native code sequence. This process is repeated for each bytecode executed. If a portion of the Java™ application is executed repeatedly, for example in a program loop, then the selection and dispatch must be performed repeatedly for each bytecode executed.
The Java™ interpreter retrieves a bytecode from an application, translates the bytecode into native machine code, and then executes the native machine code. This process is repeated for each bytecode executed. If a portion of the Java™ application is executed repeatedly, for example in a program loop, the bytecodes must be re-translated each time they are executed. Thus, although this type of Java™ virtual machine is relatively simple to implement, it is slow and inefficient.
A faster and more efficient type of Java™ virtual machine uses a Java™ bytecode compiler, rather than a Java™ interpreter. With a Java™ bytecode compiler, groups of bytecodes are converted into native machine code. The application's machine code is then executed as a group. Once the bytecodes have been translated into native machine code, they may be executed many times without re-translation. This eliminates wasteful re-translation.
FIG. 1 shows an example of a Java™ virtual machine using a Java™ bytecode compiler. Java™ virtual machine 100 receives a Java™ application in the form of a group of Java™ bytecodes 105. The Java™ bytecodes 105 are then translated into application machine code by either the Ahead-of-Time (AOT) compiler 110 or the Just-In-Time (JIT) compiler 112. Application machine code can be executed by the native system without any further translation. AOT compiler 110 translates all of the bytecodes of the Java™ application 105 into application machine code prior to the start of execution of the application. Once the AOT compiler completes the translation, the application is executed as application machine code without any further translation. JIT compiler 112 translates portions of the application's bytecodes, typically corresponding to a single method of the application, as they are encountered for the first time. Once a portion of the application has been translated, it can be executed any number of times without additional translation.
Regardless of the type of compiler employed, the application bytecodes 105 are translated into application machine code 115 for execution. During execution, application machine code 115 will access memory from the pool of free memory, referred to as the heap 120. The heap 120 stores data objects associated with the execution of the application. A garbage collector searches the heap 120 for inactive data and reclaims that space.
Java™ threads 123 execute application machine code 115. Java™ threads 123 enable multitasking within the Java™ virtual machine 100 and are discussed in detail below. Each thread is a separate stream of execution of a portion of the application machine code 115.
In addition to the application machine code 115, many Java™ applications require the use of native methods. Native methods perform functions specific to the native system. System I/O functions, which are required by many Java™ applications, are examples of functions performed with native methods. Native methods may be incorporated directly into an application, or required as part of a library, a collection of ready-made software functions that is linked to the application. For example, the Java™ Native Interface (JNI) is a standard programming interface for writing native methods and embedding the Java™ virtual machine into native applications. JNI is described in the “Java™ Native Interface Specification,” located JNI is described in the “Java™ Native Interface Specification,” located at java.sun.com/j2se/1.4.2/docs/guide/jni/spec/jniTOC.html and incorporated herein by reference.
Native methods may be written in many programming languages, for example C or C++, and are generally optimized for a specific native system. In order to be used by the Java™ virtual machine 100, libraries of native methods 117 are translated from their original programming language into native method machine code 119. This translation can be done with a compiler that is separate from the Java™ virtual machine 100.
When a native method is called by the application machine code, Java™-to-native transfer code 107 handles the transfer of execution between the application machine code 115 and the native method machine code 119. During execution of native method machine code 119, the application uses one or more native threads 133. Each native thread is a separate stream of execution with the native system. Similar to Java™ threads, native threads run in parallel, allowing multitasking in the native system. The native threads are managed by the native system. The Java™ virtual machine 100 includes interfaces allowing native method machine code 119 direct access to data stored in the virtual machine format. As it is executed by native threads 133, native method machine code 119 can access information and objects stored in heap 120 through native handle table 140. Native handle table 140 allows native method machine code to access heap 120 in coordination with the management of heap 120 by the Java™ virtual machine 100.
Many applications perform several different computing tasks in parallel using multitasking. Threading is a method that allows multitasking within a single application. Each thread is a separate stream of execution performing all or a portion of the application. Although Java™ threads and native threads have similar functions, each operates independently of the other. Java™ threads allow multiple execution streams of the application machine code, while native threads allow multiple execution streams of native methods and other native system functions.
Threads are executed concurrently, either by executing each thread on a different processor or by executing each thread for a short period of time on the same processor. Since the Java™ virtual machine is actually a software program executing on top of a native system, Java™ threads are actually executed within native threads running underneath the Java™ virtual machine. Java™ thread manager 125 determines how Java™ threads are distributed among one or more native threads. Common approaches for implementing Java™ threads include green threads, OS or native threads, and multiplexed threads.
Green threads are the simplest implementation of Java™ threads. In a green thread implementation, all of the Java™ threads run for a short period of time within a single native thread. Java™ thread manager 125 selects a Java™ thread to run in the native thread, sets a timer for a short interval, and starts the thread. When the timer expires, the Java™ thread manager stops the native thread and assigns another Java™ thread to the native thread. Despite its simplicity, the single threaded nature of a green threads implementation leads to several significant disadvantages. Since a green threads implementation only uses a single native thread, it does not take advantage of multiple processors and scales very poorly. Additionally, native methods are typically executed in the same native thread as the Java™ threads. If a native method blocks or does not return, which is common with native methods such as I/O functions, all of the Java™ threads halt execution as well, paralyzing the application.
Because of their simplicity, green threads were implemented in early versions of the Sun Java™ Development Kit (JDK), version 1.0. Green threads have been discontinued from the Microsoft Windows edition of the Sun JDK since version 1.2.0, but they remain available in many unsupported Unix editions. Additionally, the Python programming language can use a green thread implementation.
Unlike green threads, OS threads implement each Java™ thread as a separate, unshared, native thread. Since each Java™ thread is actually a native thread, the native system handles all of the management and scheduling associated with threads. Java™ thread management is delegated to the native thread management of the native system. By assigning each Java™ thread into a separate native thread, the OS threads implementation automatically take advantage of multiple processors when possible. Also, if a native method causes a native thread to block or not return, the other Java™ threads continue unimpeded.
Despite solving some of the problems associated with green threads, the OS thread implementation is slower and more inefficient. Many operating systems have technical features, such as address space restrictions, that limit the maximum number of native threads. Since each Java™ thread requires a separate native thread, this limits the maximum number of Java™ threads to the maximum number of native threads. Additionally, each native thread consumes a certain amount of native system resources as overhead. The overhead required by native threads is typically greater than that required by Java™ threads. Further, the extra overhead associated with native threads slows down transfers between Java™ threads. These factors increase the system requirements and decrease the efficiency of OS thread implementations in comparison with other Java™ thread implementations.
OS thread implementations are sometimes referred to as kernel threads or lightweight processes (LWP). OS threads are the default thread implementation for all Java™ systems released by Sun and IBM since version 1.1 of the JDK. OS thread implementations are also used by multithreaded C and C++ compilers, such as the Microsoft C and C++ compilers for Microsoft Windows.
A multiplexed thread implementation is a combination of the green thread and OS thread systems. Like a green threads system, a Java™ thread manager 125 coordinates the execution of Java™ threads. However, instead of using a single native thread, the Java™ thread manager 125 maintains a pool of a predetermined number of native threads. The number of native threads is selected to optimize performance on the native system. Java™ threads are multiplexed within the available native threads. If the number of Java™ threads is less than or equal to the number of available native threads, each Java™ thread is assigned to a separate native thread. If there are more Java™ threads than native threads, the Java™ thread manager selects Java™ threads to run for a short period of time within the available native threads. The use of multiple native threads takes advantage of multiple processors. Sharing each native thread between several Java™ threads avoids the limits on the number of native threads. Further, since thread transfers take place between Java™ threads, rather than native threads, performance is improved.
Multiplexed threads are also referred to as lightweight threads. The Unix operating system has a multiplexed threading implementation known as quickthreads. C, C++, and Python can also employ multiplexed threads.
Native method execution can present problems in multiplexed thread implementations. First, native methods may not return in a timely fashion. This can be due to a time-consuming operation, a thread blocking (being forced to wait on a resource), or the thread not returning at all. Although a delayed native method return does not paralyze the application as with green thread implementations, the pool of available threads is slowly depleted by this behavior. This can dramatically decrease the performance of the application.
Second, some native methods need to observe their thread state or thread identity to function properly. A native method that expects to be executed in the same thread on a subsequent call is an example of this. In a multiplexed thread implementation, a native method may be assigned to a different native thread each time it is called. This type of native method will not function properly in a multiplexed thread implementation.
Third, some native methods may modify a pre-existing thread state. As an example, a native method may incorrectly assume that it is the only method executing in a native thread and can reset the thread's I/O state. This will interfere with the pending I/O operations of other native methods which share this native thread.
Prior multiplexed thread implementations handled these problems with native methods in one of three ways. First, a thread manager can maintain a larger thread pool to allow for the loss of some native threads to blocking or non-returning native methods. Second, the thread manager can monitor the thread pool and add or subtract native threads as native methods change the number of available native threads. Third, the thread manager can allocate a second, separate, unshared pool of native threads for native method execution, and use the first pool of native threads for executing Java™ threads only.
Although each of these approaches solves the problem of blocking or non-returning native threads, these solutions do not scale well. Platform I/O, including network I/O, is typically implemented using native code. With large server applications, thousands of simultaneous connections may be required. It is very inefficient to allocate thousands of extra native threads to properly handle native methods. Additionally, none of these techniques address the problems caused by native methods that observe their thread state or identity, or that modify a pre-existing thread state.
Thus, it is desirable to have a threading implementation that efficiently handles native methods that: may block or fail to return in a timely fashion; may observe their OS identity or their thread state; or may modify the pre-existing thread state. It is further desirable to have a thread implementation that efficiently uses native system resources by minimizing the required number of native threads; improving the performance of thread-to-thread transfers; minimizing the overhead associated with each native thread; and taking maximum advantage of the multiprocessor capabilities of the native system.